CAMELOT: Technology Focused Testing of CSCW Applications

In this paper we describe CAMELOT, a novel technologyfocused methodology for testing collaborative software that contrasts with existing broad-based CSCW evaluation approaches. CAMELOT is intended for use by application developers, user interface specialists, performance engineers, and quality assurance personnel. The evaluation of a CSCW application is divided into two stages: single user and multiuser. The single user stage is subdivided into general computing and human-computer interaction testing. The multiuser stage is decomposed into distributed computing and human-human interaction testing. The methodology provides a detailed, codified, checklist of testing techniques for each stage. We applied CAMELOT to a conventionally tested, mature CSCW application. Our techniques uncovered and classified over two dozen problems with the system.